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Remarks 

Firstly, the current position with regard to the power of attorney is not understood. The 
assignees appointed the undersigned on 02/03/2005, and a notice indicating a change was 
mailed on 06/22/2005. A second notice was issued apparently restoring the original agent 
as a result of a power of attorney allegedly filed 04/06/2005 by the assignee. The 
undersigned has contacted the assignee who purports to have no knowledge of such a 
subsequent appointment. A further power of attorney in the undersigned's favor will be 
submitted shortly. 

The applicants have attended to the matters raised in the Notice of Non-compliance. 
While the applicants recognize the need for conformity with the USPTO rules, it is 
respectfully submitted that the objections at least based on the misplacement of the period 
before instead of after the deleted subject matter and the clear typographical error 
(missing -cd) in the status identifier of claim 21 must surely constitute an all time record 
in the practice of pedantry. Clearly the period belongs at the end of the text, and whether 
it is shown at the beginning or end of the deleted text is of no material consequence. 
Surely, there must be scope for the Examiner to exercise a certain amount of discretion 
and common sense in the implementation of the rules. 

Both these "errors" have been corrected. The bracket facing the wrong way in the 
amended paragraph in the specification was unintentional. That typographical error has 
also been corrected. 

With regard to the alleged non-compliance of the drawing amendments, it is noted that 
the drawings are line drawings corresponding to the screen shots. The Examiner was 
correct with regard to Figure 3b, in that the top of the image should coincide with the 
dashed line. This has been corrected. In Figure 3c, the solid line representing field of view 
should be a dashed line as was the case in the original Figure 3c, and this has also been 
corrected. 

In Figures 8 and 10, the edge of the image substantially coincides with the dashed line, 
which represents the field of view. While this edge is apparent in the screenshot, which 
contrasts the image with the background it is believed clearer to omit it in the line 
drawing, since the edge of the screen would be contiguous with the dashed line in such a 
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way that the dashed line would no longer be visible. Any discrepancy between the newly 
submitted drawings and the original drawings is a result of an attempt to present the 
drawings as line drawings rather than photographs. 

With regard to the substantive objections raised in the official action, the drawing figures 
have been corrected to show the dashed lines more clearly. However, the Examiner's 
objection with regard to reference numerals 24, 25, and 26 is respectfully traversed since 
these appear in the description in paragraph 42 at the top of page 8. 

Paragraph 57 has been amended to correctly identify the operator's head 105. 

It is noted that an information disclosure statement was submitted with the response filed 
on 04/13/2006. It is requested that such statement be taken into consideration by the 
Examiner. 

Claim 1 has been amended to better define the invention and overcome the informalities 
noted by the Examiner. Similar amendments have been made to claim 1 8. 

Claims 7 to 9 have been re- worded in method format. Claim 10, which is more properly 
an apparatus claim, has been made dependent on claim 18. 

With regard to claim 17, it is respectfully submitted that this is clear as amended. A 
captured image may have some degree of perspective distortion, as would be recognized 
by one skilled in the art. Claim 17 merely calls for the perspective distortion present to be 
reduced using the range data. Thus, the base level is the distortion present in the raw 
captured data. 

Claim 22 now specifies that the de-emphasis is relative to current image data (see 
paragraph 690. 

A fundamental difference between the present invention is that in the present invention 
the images are sent to the processor for immediate display , either of the direct image if 
there is no offset between the current field of view of the HMD and the camera's field of 
view at the time the image was captured, and of a transformed image corresponding to the 
overlapping portion of the two fields of view if an offset is present. In Hirose, a virtual 
omnimax image is built up, and the HMD derives its data primarily from the virtual 
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omnimax image, with a provision (see 2.3) to preferentially refresh the image in the 
current direction of the HMD. 

The significance of this difference is that Hirose, while concerned with the same problem 
of minimizing latency, is only applicable to static systems. His system is not applicable 
situations where the camera is in motion, for example as might be found on the space 
shuttle, since the surrounding scene is constantly changing. Therefore, the imagery in his 
virtual omnimax would never correspond to the actual position of the camera. 

Claims 1 and 18 are distinguished in many respects from the teachings of Hirose. For 
example, Hirose does not teach tagging the images with the CMD data, which is essential 
in the invention for the processor to be able to determine whether there is an offset 
between the HMD position and the camera position associated with the currently 
displayed data. Hirose does not teach sending a sequence of video images to the processor 
for display under the circumstances claimed, neither does he teach comparing the HMD 
position data with the CMD data for each captured image to determine whether an offset 
exists. 

Unlike Hirose, which is not truly real time because it relies on continuously scanning a 
surrounding space to build up a virtual omnimax, the present invention is a real time 
system wherein the operator can feel as if he is viewing a scene at a remote location, even 
if the camera is moving . It turns out that a short delay in the equipment responding to 
head movement is very nauseating for the operator. By creating an immediate response, 
the invention solves this problem, and unlike Hirose, solves it in a way that works when 
the camera is in motion. 

In section 2.3, Hirose briefly discusses "transmission of real-time information". He 
recognizes that limitations arise in the ability of the system to provide real-time 
information "due to capacity limitations of the system". His data always comes initially 
from the virtual dome, which is stored in computer memory. His solution is to 
preferentially refresh the images in the direction in which the user is looking. He says that 

"when a user changes his head orientation, he will initially see previously 
obtained images for a few instances. However, if the user continues looking in the 
same direction for a noticeable period of time, images from the remote camera 
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will refresh the old ones and portray more recent images of the remote location. 
Using this approach, the user can essentially see live video images without losing 
his sense of orientation. 

The operative word here is essentially. The Hirose system will never be truly real time 
because of the need to refresh the images stored in the virtual dome. He suggests giving 
priority to refreshing images in the dome in the direction in which the user is looking, but 
in his system he still needs to scan the whole space to refresh the dome. "Currently, a 
system has been implemented in which entire images of the surrounding space are first 
captured by the rotating camera head and sent to the workstation." Also, Hirose states that 
"the camera head unit is located in the remote place and continually scans its surrounding 
space." See passage bridging pages 125 and 126. 

By contrast, the invention the camera tracks head movement. Hirose teaches away from 
this approach since he states (para 2. 1) that "The key principle of the Virtual Dome was 
to separate the camera head movements and the user's head movements". 

If the proposed modification or combination of the prior art would change the 
principle of operation of the prior art invention being modified, then the teachings 
of the references are not sufficient to render the claims prima facie obvious. In re 
Ratti, 270 F.2d 810, 123 USPQ 349 (CCPA 1959) 

Since the key principle of operation of Hirose is to separate the camera head movements 
and the user's head movements, whereas in the invention the opposite is the case, namely 
the camera tracks the user's head movements, it cannot be obvious to modify Hirose to 
arrive at the invention. 

In commenting on claim 10, which claims the mechanism for moving the camera to 
follow the head mounted display, the Examiner referred to section 1 . With respect the 
Examiner cannot mix apples and oranges. Section 1 refers to a conventional prior art 
system, wherein movement of the head mounted display controls movement of the 
camera. Such a prior art system suffers from the disorientation problem the invention 
solves because of the delay in the camera tracking the operator's head movement. There 
is no disclosure of such a system in combination with any means to compensate for 
motion delay. Hirose then goes on in section 2 to say that his principle is NOT to have the 
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camera track the HMD movement but to separate the two. He then goes on to discus how 
to achieve this as discussed above. Hirose cannot therefore be regarded as a teaching of a 
HMD tracking system in combination with some system for storing or transforming 
images because Hirose expressly teaches that unlike the prior art his system does not 
track head movement. There is no logical way that Hirose's dome could be used with a 
head tracking system. Such a combination is directly contrary to his teaching. 

The prior art has to be read for what it teaches as a whole. It is not permissible just to 
extract portions of a document and piece them together in a way that is directly contrary 
to the teaching of the document. To do that is to make a hindsight reconstruction of the 
invention using the applicant's own teachings. 

prior art reference must be considered in its entirety, i.e., as a whole, including 
portions that would lead away from the claimed invention. W.L. Gore & 
Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), 
cert, denied, 469 U.S. 851 (1984) 

Moreover, there is no teaching of associating the actual images with the CMD data as 
claimed and comparing the CMD data with the current HDM date to determine whether 
an image transformation is necessary. 

Claims 1 and 18 are therefore believed patentable, as are the remaining claims which are 
dependent thereon. 

Allowance and reconsideration are therefore earnestly solicited. 
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Respectfully submitted, 



Registration No. 34519 
Richard J. Mitchell 
Agent of Record 

MARKS & CLERK 

P. O. Box 957, Station B, 

Ottawa, Ontario, Canada 

K1P5S7 

(613)236-9561 




page 15 of 15 



